Immersive audio reproduction systems

ABSTRACT

Systems and methods can provide an elevated, virtual loudspeaker source in a three-dimensional soundfield using loudspeakers in a horizontal plane. In an example, a processor circuit can receive at least one height audio signal that includes information intended for reproduction using a loudspeaker that is elevated relative to a listener, and optionally offset from the listener&#39;s facing direction by a specified azimuth angle. A first virtual height filter can be selected for use based on the specified azimuth angle. A virtualized audio signal can be generated by applying the first virtual height filter to the at least one height audio signal. When the virtualized audio signal is reproduced using one or more loudspeakers in the horizontal plane, the virtualized audio signal can be perceived by the listener as originating from an elevated loudspeaker source that corresponds to the azimuth angle.

CLAIM OF PRIORITY

This patent application is a Continuation of U.S. patent applicationSer. No. 15/587,903, filed on May 5, 2017, which claims the benefit ofpriority to U.S. Provisional Patent Application No. 62/332,872, filed onMay 6, 2016, the contents of which are incorporated by reference hereinin their entireties.

BACKGROUND

Various techniques have been proposed for implementing audio signalprocessing based on Head-Related Transfer Functions (HRTF), such as forthree-dimensional audio reproduction using headphones or loudspeakers.In some examples, the techniques are used for reproducing virtualloudspeakers localized in a horizontal plane, or located at an elevatedposition. To reduce horizontal localization artifacts for listenerpositions away from a “sweet spot” in a loudspeaker-based system,various filters can be applied to restrict the effect to lowerfrequencies. However, this can compromise an effectiveness of a virtualelevation effect.

Such techniques generally require or use an audio input signal thatincludes at least one dedicated channel intended for reproduction usingan elevated loudspeaker. However, some commonly available audio content,including music recordings and movie soundtracks, may not include such adedicated channel. Using a “pseudo-stereo” technique to spread an audiosignal over two loudspeakers is generally insufficient or not suitablefor producing a desired vertical immersion effect, for example, becauseit vertically elevates and expands the reproduced audio image globally.For a more natural-sounding immersion or enhancement effect, it isdesirable to preserve the perceived localization of primary signalcomponents (e.g., in the horizontal plane), while providing a perceivedvertical expansion for ambient or diffuse signal components.

In an example, an upward-firing loudspeaker driver can be used toreflect height signals on a listening room's ceiling. This approach isnot always practical, however, because it requires a horizontal ceilingat a moderate height, and calls for additional system complexity forcalibration and relative delay alignment of height channel signals withrespect to horizontal channel signals.

OVERVIEW

The present inventors have recognized that a problem to be solvedincludes providing an immersive, three-dimensional listening experiencewithout requiring or using elevated loudspeakers. The problem canfurther include providing a virtual sound source in three-dimensionalspace relative to a listener, such as at a vertically elevated location,and at a specified angle relative to a direction in which the listeneris facing. The problem can include tracking movement of the listener andcorrespondingly adjusting or maintaining the virtual sound source in theuser's three-dimensional space. The problem can further includesimplifying or reducing hardware requirements for reproducingthree-dimensional or immersive sound field experiences.

In an example, a solution to the vertical localization problem includessystems and methods for immersive spatial audio reproduction.Embodiments can use loudspeakers to reproduce sounds perceived bylisteners as coming at least in part from an elevated location, such aswithout requiring or using physically elevated or upward-firingloudspeakers. Various embodiments are compatible with or selected forspecified audio playback devices including headphones, loudspeakers, andconventional stereo or surround sound playback systems. For example,some systems and methods described herein can be used for playback ofenhanced, immersive three-dimensional multi-channel audio content suchas using sound bar loudspeakers, home theater systems, or using TVs orlaptop computers with integrated loudspeakers.

Besides the hardware simplification and cost savings from eliminatingdedicated “height” loudspeakers or drivers, the present systems andmethods include various advantages. For example, the signal processingmethods can implement virtual height effects independently fromhorizontal-plane localization processing or rendering. This can permitoptimization or tuning of the vertical and horizontal aspectsseparately, thereby preserving an elevation effect even at listeningpositions away from a “sweet spot” and independent of horizontalsurround effect design compromises.

By removing dependencies between a virtual elevation effect and ahorizontal-plane localization, efficient signal processing topologiescan be enabled. In an example, the same or similar virtual height effecttopology can be used whether a system includes only a two-channel stereoloudspeaker arrangement or the system includes additional loudspeakers,such as in a multi-channel surround sound system that includes front andrear loudspeakers. In an example, a multi-channel system example can usevirtual rear elevation effects using the physical rear loudspeakers. Inanother example, a two-channel system example can use the virtual rearelevation effect in conjunction with a horizontal plane rearvirtualization. The virtual height processing topology can be the samefor both examples.

In an example, height upmixing techniques can be used to generate anenhanced immersion effect, such as for legacy content formats that maynot include discrete height channels. The height upmix techniques caninclude vertically expanding a perceived localization of ambientcomponents in input signals.

A solution to the above-described problems can include or use virtualheight audio signal processing to deliver a more accurate and immersivesound field using conventional horizontal loudspeaker or headphoneconfigurations. In an example, virtual height processing can apply avirtual height filter to audio signals intended for delivery usingelevated loudspeakers. Such a virtual height filter can be derived froma head-related transfer function (HRTF) magnitude or power ratiocharacteristic. In some examples, the HRTF magnitude or powerinformation can be derived independently of a desired azimuthlocalization angle relative to a listener's look or facing direction.The power ratio can be evaluated for a sound source located in a medianplane in front of the listener. However, this approach may not addressvirtual height processing for sound localization away from the medianplane.

In an example, virtual height processing can include or use a virtualheight filter that is dependent, at least in part, on a specifiedazimuth, or rotational direction, of a virtual sound source relative toa listener's look direction. In an example, the processing can accountfor various differences between ipsilateral and contralateral HRTFs forelevated virtual sources.

In an example, a further solution to the above-described problems caninclude or use HRTF-based virtualization of phantom sources. Phantomsources can include audio information or sound signals that areamplitude-panned between multiple input or output channels, and suchphantom sources are generally perceived by a listener as originatingfrom somewhere between the loudspeakers. In an example, virtualizationtechniques, such as include frequency-domain spatial analysis andsynthesis techniques, can be used for extracting and “re-rendering”phantom sound components at their respective proper or intendedlocalizations, and decorrelation processing can be used together withvirtualization to improve reproduction of phantom components, such asphantom center components.

In an example, a variable decorrelation effect can be incorporated in apair of digital finite-impulse-response (FIR) HRTF filters.

In some examples, decorrelation processing can be applied exclusively tophantom-center sound components and no virtualization processing isapplied to the decorrelated signals. In other examples, decorrelationprocessing can be incorporated within virtualization filters. In stillother examples, the immersive spatial audio reproduction systems andmethods described herein include or use virtualization of phantomsources, and decorrelation filters can be applied to input channelsignals, such as prior to virtualization processing.

In an example, the immersive spatial audio reproduction systems andmethods described herein can include or use low-complexity time-domainupmix processing techniques to generate an enhanced immersion effect,such as by vertically expanding a listener-perceived localization ofambient and/or diffuse components present in an input audio signal. Theenhanced immersion effect can exhibit minimal or controlled effects on alocalization of primary sound components. Upmix techniques can includepassive or active matrices, the latter including frequency-domainalgorithms (e.g., such as DTS® Neo:X™ and DTS® Neural:X™) that canderive synthetic height channels from legacy multi-channel content, suchas from 5.1 surround sound content.

It should be noted that alternative embodiments are possible, and stepsand elements discussed herein may be changed, added, or eliminated,depending on the particular embodiment. These alternative embodimentsinclude alternative steps and alternative elements that may be used, andstructural changes that may be made, without departing from the scope ofthe invention.

This overview is intended to provide an overview of subject matter ofthe present patent application. It is not intended to provide anexclusive or exhaustive explanation of the invention. The detaileddescription is included to provide further information about the presentpatent application.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. The drawings illustrate generally, by way of example, butnot by way of limitation, various embodiments discussed in the presentdocument.

FIG. 1 illustrates generally first and second examples and of audiosignal playback in a three-dimensional sound field.

FIG. 2 illustrates an example of multiple ipsilateral and contralateralelevation spectral response charts.

FIG. 3 illustrates generally first and second examples and of virtualheight and horizontal plane sound signal spatialization.

FIG. 4 illustrates generally an example of a system that uses multiplevirtual height loudspeakers to simulate an 11.1 playback system.

FIG. 5 illustrates generally an example of a virtualizer processingsystem, according to some embodiments.

FIG. 6 illustrates generally an example of a second virtualizerprocessing system, according to some embodiments.

FIG. 7 illustrates generally an example of a block diagram of a portionof a system for virtual height processing.

FIG. 8 illustrates generally an example of a block diagram of a nestedall-pass filter.

FIG. 9 illustrates generally first, second, and third examples of avirtual height processor in a 9-channel input system.

FIG. 10 illustrates generally an example of height upmix processing.

FIG. 11 illustrates generally a block diagram of height upmix processingfor a single channel input signal.

FIG. 12 illustrates generally a block diagram of an example of theDecorrelation module from the example of FIG. 11.

FIG. 13 illustrates generally a first height upmix processing example.

FIG. 14 illustrates generally a second height upmix processing example.

FIG. 15 illustrates generally a third height upmix processing example.

FIG. 16 illustrates generally a fourth height upmix processing example.

FIG. 17 illustrates generally first, second, and third examples of avirtual height upmix processor in a 5-channel input system.

FIG. 18 is a block diagram illustrating components of a machine that isconfigurable to perform any one or more of the methodologies discussedherein.

DETAIL ED DESCRIPTION

In the following description that includes examples of environmentrendering and audio signal processing, such as for reproduction viaheadphones or other loudspeakers, reference is made to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments in which theinvention can be practiced. These embodiments are also referred toherein as “examples.” Such examples can include elements in addition tothose shown or described. However, the present inventors alsocontemplate examples in which only those elements shown or described areprovided. The present inventors contemplate examples using anycombination or permutation of those elements shown or described (or oneor more aspects thereof), either with respect to a particular example(or one or more aspects thereof), or with respect to other examples (orone or more aspects thereof) shown or described herein.

As used herein, the phrase “audio signal” is a signal that isrepresentative of a physical sound. Audio processing systems and methodsdescribed herein can use or process audio signals using various filters.In some examples, the systems and methods can use signals from, orsignals corresponding to, multiple audio channels. In an example, anaudio signal can include a digital signal that includes informationcorresponding to multiple audio channels.

Various audio processing systems and methods can be used to reproducetwo-channel or multi-channel audio signals over various loudspeakerconfigurations. For example, audio signals can be reproduced overheadphones, over a pair of bookshelf loudspeakers, or over a surroundsound system, such as using loudspeakers positioned at various locationswith respect to a listener. Some examples can include or use compellingspatial enhancement effects to enhance a listening experience, such aswhere a number or orientation of loudspeakers is limited.

In U.S. Pat. No. 8,000,485, to Walsh et al., entitled “Virtual AudioProcessing for Loudspeaker or Headphone Playback”, which is herebyincorporated by reference in its entirety, audio signals can beprocessed with a virtualizer processor to create virtualized channelsignals that can be summed with other signals to produce a modifiedstereo image. Additionally or alternatively to the techniques in the'485 patent, the present inventors have recognized that virtual heightprocessing can be used to deliver an accurate sound field representationthat includes vertical components while using horizontally-arrangedloudspeaker configurations.

In an example, relative virtual elevation filters, such as can bederived from head-related transfer functions, can be applied to rendervirtual audio information that is perceived by a listener as includingsound information at various specified altitudes or elevations above orbelow a listener to further enhance a listener's experience. In anexample, such virtual audio information is reproduced using aloudspeaker provided in a horizontal plane and the virtual audioinformation is perceived to originate from a loudspeaker or other sourcethat is elevated relative to the horizontal plane, such as even when nophysical or real loudspeaker exists in the perceived originationlocation. In an example, the virtual audio information provides animpression of sound elevation, or an auditory illusion, that extendsfrom, and optionally includes, audio information in the horizontalplane.

FIG. 1 illustrates generally first and second examples 101 and 151 ofaudio signal playback in a three-dimensional sound field. In the firstexample 101, a listener 110 faces a first direction 111, or “lookdirection.” In the example, the look direction extends along a firstplane associated with the listener 110. In some examples, the firstplane includes a horizontal plane that coincides with the ears of thelistener 110, or with the torso of the listener 110, or with a waist ofthe listener 110. The first plane, in other words, can be referenced toa specified orientation or location relative to the listener 110.

FIG. 1 illustrates a virtual height processing filter from a firsthead-related transfer function (HRTF) filter H(z), such as can bemeasured at a first position 121 in a median plane relative to a head ofthe listener 110. That is, in an example, the first position 121 canhave a 0 degree azimuth angle in a horizontal, front direction withrespect to the listener 110.

In the second example 151, the listener 110 faces the first direction111, and a second virtual height processing filter from a secondhead-related transfer function (HRTF) filter H_(H)(z) can be measured ata second position 122 relative to a head of the listener 110. In thisexample, the second position 122 is provided at an elevated position inthe median plane. That is, the second position 122 can have a 0 degreeazimuth angle and a non-zero altitude angle θ in a horizontal, frontdirection with respect to the listener 110.

In the first example 101, an audio input signal, denoted X in Equation(1), below, can be provided by a loudspeaker at the first position 121in the median plane. A signal Y received at the left or right ear of thelistener 110 can be expressed as:Y(z)=H(z)X(z)  (1)

In the second example 151, a signal Y_(H) received at the left or rightear of the listener 110 can be expressed as:Y _(H)(z)=H _(H)(z)X(z)  (2)A listener's perception that signal X emanates or originates from thesecond position 122 while using a loudspeaker located at the firstposition 121 can be provided by ensuring that the reproduced audiosignal, as received by the listener 110, has substantially the samemagnitude spectrum as signal Y_(H). Such a signal can be obtained bypre-filtering the input signal X with a virtual height filter E_(H), tothereby yield a modified loudspeaker input signal X′ and a receivedsignal Y′ such that:|Y′(z)|=|H(z)| |X′(z)|=|H(z)| |E _(H)(z)X(z)  (3)and|H(z)| |E _(H)(z)X(z)|=|H(z)| |E _(H)(z)| |X(z)|  (4)

In an example, a magnitude spectrum |Y′(z)| can be made substantiallyequal to |Y_(H)(z)| for any input signal X, such as when the magnitudetransfer function |E_(H)(z)| of the virtual height filter satisfiesEquation (5).|H(z)| |E _(H)(z)|=|H _(H)(z)  (5)

In an example, the virtual height filter E_(H)(z) can be designed as aminimum-phase filter or as a linear-phase filter whose magnitudetransfer function |E_(H)(z)| is substantially equal to the magnitudespectral ratio of the HRTF filters H_(H)(z) and H(z), as shown inEquation 6.|E _(H)(z)|=|H _(H)(z)|/|H(z)|  (6)When a minimum-phase design is used, the virtual height filter E_(H)(z)can be defined as shown in Equation 7.E _(H)(z)={H _(H)(z)} {H(z)}⁻¹  (7)In Equation (7), and throughout this discussion, {G(z)} denotes aminimum-phase transfer function having magnitude equal to |G(z)|, suchas for any transfer function G(z).

FIG. 2 illustrates an example of multiple elevation spectral responsecharts. Each of the illustrated charts shows HRTF spectral ratioinformation, wherein the x axis represents frequency and the y axisrepresents a relative amplitude ratio expressed in decibels. Thespectral ratio information is for a sound source located at 45 degreeselevation and various azimuth angles (φ) or positions, includingipsilateral front and back positions, and contralateral front and backpositions. For example, FIG. 2 includes a first chart 201 that shows afirst trace 211 that indicates a frequency vs. relative amplitude ratiorelationship for an ipsilateral front position of the listener 110. Thatis, the first chart 201 indicates that different frequency-specific HRTFfilter characteristics can be used when a height or elevation of thesource is fixed (e.g., at 45 degrees) and the source is intended to beperceived as originating or including information from an ipsilateralfront position. A second chart 202 shows a second trace 212 thatindicates a frequency vs. relative amplitude ratio relationship for anipsilateral back or rear position of the listener 110. Third and fourthcharts 203 and 204 similarly show third and fourth traces 213 and 214that indicate frequency vs. relative amplitude ratio relationship forcontralateral front and contralateral back positions of the listener110, respectively.

From the example of FIG. 2, the HRTF magnitude ratio (e.g., elevationspectral cue) changes with the azimuth angle (φ) or position. Therefore,rather than keeping a virtual height filter constant, such as regardlessof an azimuth angle (φ), an effective or accurate virtual height effectcan be provided using a virtual height filter that depends at least inpart on a specified azimuth angle (φ). In an example, the virtual heightfilter can be independent of a horizontal-plane sound spatializationmethod used, such as to more closely match a measured elevation spectralcue for a given azimuth angle (φ).

FIG. 3 illustrates generally first and second examples 301 and 351 ofvirtual height and horizontal plane sound signal processing orspatialization. Such spatialization can include, for instance, amplitudepanning, Ambisonics, and HRTF-based virtual loudspeaker processingtechniques. Properly applied, these techniques can be used toapproximate signals that would be received at the ipsilateral andcontralateral sides of the listener 110, such as if the input signal Xwas played from a loudspeaker located in the soundfield at an azimuthangle φ and at an altitude angle θ.

In the first example 301, the listener 110 can face or look in a seconddirection 311 in a three-dimensional soundfield. A virtual source 305located in the soundfield can be provided at coordinates (x, y, z) in athree-dimensional sound field, such as where the listener 110 is locatedat the origin of the field. A localization problem can includedetermining which of multiple available processing or spatializationtechniques to use or apply to the input signal X such that the listener110 perceives the reproduced signal as originating from the virtualsource 305.

The second example 351 illustrates generally an example of a solution tothe localization problem that includes providing a virtual sound source.The second example 351 includes the same listener 110 facing in thesecond direction 311. To provide an auditory illusion of an elevatedsound source, such as located at a non-zero azimuth angle φ and at anon-zero altitude angle θ, such as outside of the median plane, thesecond example 351 can include pre-filtering, such as using the virtualheight filter E_(H)(z) of Equation (6) to apply horizontal-plane soundspatialization. In the example of FIG. 3, the audio input signal can befirst processed, such as using an audio processor circuit, using aHorizontal Plane Virtualization module 365 to virtualize or provide ahorizontally-located signal at coordinates (x, y). Thehorizontally-located signal can then be further processed, such as usingthe same or different audio processor circuit including a HeightVirtualization module 375 to virtualize or provide a vertically-locatedsignal at a distance z from the horizontally-located signal. That is, inan example, an audio processor circuit can be used to generate avirtualized or localized height audio signal such as by applying signalfilters (e.g., HRTF-based filters) to one or more source signals.Although FIG. 3 depicts the vertically-located signal as being elevatedrelative to the plane of the listener 110, the vertically-located signalcould alternatively or additionally be lowered relative to the plane ofthe listener 110.

Virtualization techniques described herein can be used or applied tosimulate different playback system configurations. FIG. 4, for example,illustrates generally an example of a system 400 that can include or usemultiple virtual height loudspeakers to simulate an 11.1 surround soundplayback system. For example, the system 400 can include a 7.1horizontal surround sound playback system with four virtual heightloudspeakers to provide or simulate an 11.1 (or 7.1.4) playback systemfor the listener 110. In the example of the system 400, the horizontalsurround sound playback system includes at least a center speaker 401,left front speaker 402, right front speaker 403, left side speaker 404,right side speaker 405, left rear speaker 406, and right rear speaker407. In an example, any one or more of the speakers in the system 400are virtualized except for the left front speaker 402 and the rightfront speaker 403.

In the example of FIG. 4, the system 400 includes a virtual left frontheight speaker 412, a virtual right front height speaker 413, a virtualleft rear height speaker 416, and a virtual right rear height speaker417. In an example, each virtual height loudspeaker can be providedusing a horizontal-plane physical loudspeaker or horizontal-planevirtual loudspeaker having the same or similar azimuth angle, and thatreceives for reproduction a signal that is pre-filtered with a virtualheight filter that is configured to simulate the elevation spectral cuecalculated for the specified azimuth angle (see, e.g., the charts201-204 from the example of FIG. 2 showing examples of differentelevation spectral cues). In an example, a magnitude transfer functionof a virtual height filter for each azimuth angle can be calculated bypower averaging of the ipsilateral and contralateral HRTFs prior tocomputing the spectral magnitude or power ratio at each frequency.

FIG. 5 illustrates generally an example of a virtualizer processingsystem 500, according to some embodiments. In the example, thevirtualizer processing system 500 includes a horizontal-planevirtualizer circuit 501 (e.g., corresponding to the Horizontal PlaneVirtualization module 365) configured to receive a horizontal audiosignal input pair (signals designated L and R) and provide an outputpair, such as to a corresponding pair of output loudspeaker drivers orto an amplifier circuit. The system 500 further includes a heightvirtualizer circuit 502 (e.g., corresponding to the HeightVirtualization module 375) configured to receive a height audio signalinput pair (signals designated Lh and Rh).

In the example of the system 500, the horizontal-plane virtualizercircuit 501 provides horizontal-plane spatialization to the audio signalinput pair (L, R). In an example, the horizontal-plane virtualizercircuit 501 is realized using a “transaural” shuffler filter topologythat assumes that the L and R virtual loudspeakers are symmetricallylocated relative to the median plane, as well as to the two outputloudspeaker drivers. Under this assumption, the sum and differencevirtualization filters can be designed according to Equations 8 and 9:H _(SUM) ={H _(i) +H _(c) } {H _(0i) +H _(0c)}⁻¹  (8)H _(DIFF) ={H _(i) −H _(c) } {H _(0i) −H _(0c)}⁻¹  (9)In Equations 8 and 9, dependence on the frequency variable z is omittedfor simplification, and the following HRTF notations are used:

-   H_(0i): ipsilateral HRTF for a left or right physical loudspeaker    location;-   H_(0c): contralateral HRTF for a left or right physical loudspeaker    location;-   H_(i): ipsilateral HRTF for a left or right virtual loudspeaker    location; and-   H_(c): contralateral HRTF for a left or right virtual loudspeaker    location.

In an example, by replacing in Equations (8) and (9) the horizontal HRTFpair (H_(i); H_(c)) with a height HRTF pair (e.g., H_(Hi) and H_(Hc),wherein H_(Hi) is an ipsilateral HRTF for the left or right virtualheight loudspeaker locations, and H_(Hc) is a contralateral HRTF for theleft or right virtual height loudspeaker locations), the samevirtualizer processing system 500 topology can be used to simulate orvirtualize height loudspeakers in order to reproduce the height channelsignals Lh and Rh.

In some examples, virtual height loudspeakers can be simulated as shownin FIG. 5 using pre-processing of the height audio signal input pairsignals Lh and Rh with the virtual height filter E_(H), such as prior tohorizontal-plane virtualization processing. In an example, this approachcan be advantageous because it can help reduce a computational load onthe system 500, such as by sharing a single horizontal virtualizationprocessing block for the audio signal input pair (L, R) and the heightaudio signal input pair (Lh, Rh). In an example, pre-processing theheight audio signal input pair signals can help preserve a subjectiveeffectiveness of the virtual height filter, such as independently of thefilter design optimizations that may be applied by the horizontal planevirtualizer circuit 501.

In an example, the elevation filter E_(H) can be incorporated directlywithin the sum and difference filter pair (H_(SUM); H_(DIFF)) byreplacing it with (E_(H)H_(SUM); E_(H)H_(DIFF)). Therefore, in avirtualizer design where H_(SUM) and H_(DIFF) are band-limited to lowerfrequencies, or otherwise modified from Equations (8) and (9), aneffectiveness of the virtual height effect can be independentlycontrolled.

FIG. 6 illustrates generally an example of a second virtualizerprocessing system 600, according to some embodiments. In the example,the second virtualizer processing system 600 includes thehorizontal-plane virtualizer circuit 501, such as configured to receivea horizontal audio signal input pair (signals designated L and R) andprovide an output pair, such as to a corresponding pair of outputloudspeaker drivers or to respective channels in an amplifier circuit.The system 600 further includes a second height virtualizer circuit 602configured to receive a height audio signal input pair (e.g., signalsdesignated Lh and Rh).

In the example of FIG. 6, the second virtualizer processing system 600can be configured to differentiate reproduction of ipsilateral andcontralateral elevation spectral cues. In this example, the virtualheight loudspeaker signals Lh and Rh can be assumed to be symmetricallylocated relative to the median plane, and the second height virtualizercircuit 602 includes a sum filter and a difference filter, wherein:E _(SUM,H) ={H _(Hi) +H _(Hc) } {H _(i) +H _(c)}⁻¹  (10)E _(DIFF,H) ={H _(Hi) −H _(Hc) } {H _(i) −H _(c)}⁻¹  (11)

In other examples for virtual loudspeaker processing, virtual heightprocessing can be incorporated directly within the sum and differencefilter pair (H_(SUM); H_(DIFF)) such as by replacing it with (E_(SUM,H)H_(SUM); E_(DIFF,H) H_(DIFF)). Thus in a system where H_(SUM) andH_(DIFF) are band-limited to lower frequencies or otherwise modifiedfrom Equations (8) and (9), an effectiveness of a virtual height effectcan be independently controlled.

In an example, virtual height processing can be applied to multi-channelsignals. Multi-channel audio signals can include sound components thatare “panned” across two or more audio channels in order to provide soundlocalizations that do not coincide with static or physical loudspeakerpositions. Such panned sounds can be referred to as “phantom sources”.

Referring again to FIG. 4, the system 400 illustrates first and secondvirtual phantom sources 421 and 422. In an example, an input signalpanned between the front left and right height input channels providesthe first virtual phantom source 421. When these input channels arereproduced as virtual loudspeakers, the perceived result is referred toas a virtual phantom source. Similarly, the second virtual phantomsource 422 can represent a localization such as after virtualloudspeaker processing for a phantom source panned between the frontright height and rear right height input channels.

Even when virtual loudspeaker processing faithfully reproduceslocalization effects of each input channel signal auditionedindividually, it can be observed that a rendering of virtual phantomsources can suffer audible degradation in localization, loudness ortimbre when combined with other corresponding audio program material.For example, a perceived localization of the first virtual phantomsource 421 can be less elevated than expected, such as compared to thevirtual left front height speaker 412 and the virtual right front heightspeaker 413. In some examples, this degradation issue can be mitigatedby applying inter-channel decorrelation processing, such as prior tovirtualization processing.

FIG. 7 illustrates generally an example of a block diagram of a portionof a system 700 for virtual height processing. In an example, the system700 is configured to receive a 4-channel input signal comprising a frontheight input signal pair (Lh, Rh) and a rear or side height input signalpair (Lsh, Rsh). The system includes a Decorrelation module configuredto apply a decorrelation filter to each of the input signals separately.In an example, the Decorrelation module applies a respective differentall-pass filter to each of the input signals, and the each of thefilters can be differently configured.

Decorrelation is an audio processing technique that reduces acorrelation between two or more audio signals or channels. In someexamples, decorrelation can be used to modify a listener's perceivedspatial imagery of an audio signal. Other examples of usingdecorrelation processing to adjust or modify spatial imagery orperception can include decreasing a perceived “phantom” source effectbetween a pair of audio channels, widening a perceived distance betweena pair of audio channels, improving a perceived externalization of anaudio signal when it is reproduced over headphones, and/or increasing aperceived diffuseness in a reproduced sound field.

In an example, a method for reducing correlation between two (or more)audio signals includes randomizing a phase of each audio signal. Forexample, respective all-pass filters, such as each based upon differentrandom phase calculations in the frequency domain, can be used to filtereach audio signal. In some examples, decorrelation can introduce timbralchanges or other unintended artifacts into the audio signals.

In the example of FIG. 7, the various input signals can receivedecorrelation processing prior to virtualization, that is, prior tobeing subjected to any virtual height filters or spatial localizationprocessing. After decorrelation processing, the input signals (e.g.,source signals panned between the Lh and Rh input channels) can be madeto be heard by the listener at virtual positions substantially locatedon the shortest arc centered on the listener's position and joining thedue positions of the virtual loudspeakers. The present inventors haverecognized that such decorrelation processing can be effective inhelping to avoid various virtual localization artifacts, such as in-headlocalization, front-back confusion, and elevation errors, such as candetract from a listener's experience.

FIG. 8 illustrates generally an example of a block diagram of a nestedall-pass filter 800. Filter parameters M, N, g1, and g2 influence adecorrelation effect of the filter 800, such as relative to othersignals processed using other filters or using another instance of thefilter 800 with different parameters. In an example, each decorrelationfilter from the system 700 of FIG. 7 includes an instance of the nestedall-pass filter 800 from the example of FIG. 8.

In an example, inter-channel decorrelation can be obtained by choosingdifferent values for the parameters M, N, g1 and g2 of each nestedall-pass filter (as represented by different letters A, B, C, and D inthe example of FIG. 7). Other decorrelation filter types or techniquescan similarly be used in the Decorrelation block of the system 700.

Referring again to FIG. 7, the system 700 further includes a VirtualHeight Filter module. In the Virtual Height Filter module, a respectivevirtual height filter can be applied to each of the four input signals(Lh, Rh, Lsh, Rsh). In the example, each filter is modeled as a seriesor cascade of second-order digital IIR filter sections. Other digitalfilter implementations can be based on specified magnitude or frequencyresponse characteristics and can be used for virtual height filters. Inthe example of FIG. 7, a Surround Processing module follows the VirtualHeight Filter module. In an example, the Surround Processing moduleincludes a front-channel horizontal-plane virtualizer applied to thefront height input signal pair (Lh, Rh) (see, e.g., FIG. 5), and arear-channel horizontal-plane virtualizer applied to the rear heightinput signal pair (Lsh, Rsh).

FIG. 9 illustrates generally first, second, and third examples 901, 902,and 903, of a virtual height processor in a 9-channel input system. Thefirst example 901 includes a signal flow diagram showing a 9-channelinput signal 911 that includes signal components or channels L, R, C,Ls, Rs, Lh, Rh, Lsh, and Rsh. Various hardware circuitry can be used toreceive the 9-channel input signal 911, such as including discreteelectrical or optical input paths to receive time-varying audio signalinformation at an audio processor circuit.

In an example, one or more of the signal components or channels includesmetadata (e.g., analog or digital data encoded with audio signalinformation) with information about a localization for one or more ofthe same or other signal components or channels. For example, the leftheight channel Lh and the right height channel Rh can include respectivedata or information about a specified localization of the audio contentincluded therein. In an example, the localization information can beprovided via other means, such as using a separate or dedicated hardwareinput to an audio processor circuit. The localization information caninclude an indication as to which channel(s) the localizationinformation corresponds. In an example, the localization informationincludes azimuth and/or altitude information. The altitude informationcan include an indication of a localization that is above or below areference plane.

In the first example 901, height-channel input signals Lh, Rh, Lsh, andRsh are provided to a Decorrelation module 912 where one or more of thefour input signals is subjected to a decorrelation filter. In anexample, each of the four input signals is subject to a decorrelationfilter that includes or uses a nested all-pass filter, such as thefilter 800 of FIG. 8. In an example, each of the four input signals issubjected to a different instance of the decorrelation filter anddifferent decorrelation filter parameters are used for each instance.The Decorrelation module 912 can include or use other circuits (e.g.,high pass, low pass, or other filters) to decorrelate the input signals.

Following decorrelation processing by the Decorrelation module 912,resulting decorrelated signals are provided to a Virtual Height Filtermodule 913. In an example, the Virtual Height Filter module 913 includesor uses the Height Virtualization module 375 from the example of FIG. 3and applies signal processing or filtering to the one or moredecorrelated signals to provide a virtualized height audio informationsignal. At the Virtual Height Filter module 913, a front virtual heightfilter can be selected and applied to the height audio signal input pair(Lh, Rh), such as described above in the discussion of FIG. 5. In anexample, the front virtual height filter is selected using a processorcircuit to retrieve an appropriate filter based on an azimuth parameterassociated with the input signal(s). In an example, a rear virtualheight filter can be applied to the rear height input signal pair (Lsh,Rsh). In some examples, the front and rear virtual height filters can bebased on azimuth angle-specific HRTF data, such as can be measuredrelative to the direction of the C-channel (e.g., front center) speaker.Following the Virtual Height Filter module 913, filtered signals can beprovided to a Mixer module 914, and the filtered height signals Lh, Rh,Lsh and Rsh can be down-mixed into the corresponding horizontal inputsignal (respectively L, R, Ls and Rs) to produce a 5-channel outputsignal 920. That is, the Mixer module 914 can provide means or hardwarefor combining or summing one or more components of a virtualized heightaudio information signal (e.g., from the virtual height filter 913) withone or more other signals (e.g., from the 9-channel input signal 911)that are configured or desired to be concurrently reproduced. In anexample, the 5-channel output signal 920 can be configured for use inaudio reproduction using loudspeakers in a first plane of a listener toproduce audible information that is perceived by the listener asincluding information outside of the first plane, for example, above orbelow the first plane.

The second example 902 of FIG. 9 includes a signal flow diagram showingthe 9-channel input signal 911 that includes signal components orchannels L, R, C, Ls, Rs, Lh, Rh, Lsh, and Rsh. In the second example902, the height-channel input signals Lh, Rh, Lsh, and Rsh are providedto the Decorrelation module 912 and to the Virtual Height Filter module913, similarly to the first example 901. Following the Virtual HeightFilter module 913, filtered signals can be provided to a Mixer module924, and the filtered height signals Lh, Rh, Lsh and Rsh can bedown-mixed into the corresponding horizontal input signal (respectivelyL, R, Ls and Rs) to produce a 5-channel output signal. In the secondexample 902, the 5-channel output signal can be further processed by aHorizontal Surround Processing module 925 configured to provide atwo-channel loudspeaker output signal 926. The two-channel output signal926 can be configured for use in audio reproduction using loudspeakersin a first plane of a listener to produce audible information that isperceived by the listener as including information outside of the firstplane, for example, above or below the first plane. In some examples,the Surround Processing module 925 includes a front-channelhorizontal-plane virtualizer applied to a front signal pair (L, R), suchas shown in FIG. 5, and a rear-channel horizontal-plane virtualizerapplied to a side signal pair (Ls, Rs) In an example, the HorizontalSurround Processing module 925 can include or use the Horizontal PlaneVirtualization module 365 from the example of FIG. 3 to virtualize orprovide horizontally-located signal components.

The third example 903 of the example of FIG. 9 includes a signal flowdiagram showing the 9-channel input signal 911 that includes signalcomponents or channels L, R, C, Ls, Rs, Lh, Rh, Lsh, and Rsh. In thethird example 903, the height-channel input signals Lh, Rh, Lsh, and Rshare provided to the Decorrelation module 912 and the Virtual HeightFilter module 913, similarly to the first example 901. In an example,the Virtual Height Filter module 913 can be configured to down-mix thefiltered signals to a signal pair and provide the signals to a HeightSurround Processing module 931. Horizontal input signals L, R, C, Ls,and Rs, can be separately processed using a Horizontal SurroundProcessing module 932. In an example, the Horizontal Surround Processingmodule 932 can include or use the Horizontal Plane Virtualization module365 from the example of FIG. 3 to virtualize or providehorizontally-located signal components. Outputs from the Height SurroundProcessing module 931 and the Horizontal Surround Processing module 932can be provided to a Mixer module 934 that is configured to further mixthe signals and provide a two-channel loudspeaker output signal 936. Inan example, the two-channel output signal 936 can be configured for usein audio reproduction using loudspeakers in a first plane of a listenerto produce audible information that is perceived by the listener asincluding information outside of the first plane, for example, above orbelow the first plane.

In an example, an input signal intended for presentation or reproductionusing a loudspeaker in a horizontal plane can be modified to derive anoutput signal that is to be provided to a real or virtual heightspeaker. Such input signal processing can be referred to as heightupmixing or height upmix processing.

FIG. 10 illustrates generally an example of height upmix processing.FIG. 10 includes a first example 1001 wherein an apparent sound sourcelocation 1010 is spaced from the listener 110. In an example, anintended effect of height upmix processing is to vertically expand aperceived extent of diffuse sounds, such as while maintaining aperceived sound source localization, such as in a horizontal plane. FIG.10 further includes a second example 1051 wherein the apparent soundsource location 1010 remains at substantially the same azimuth angle butwith an apparent vertical extension of diffuse sounds to provide asignal for a height speaker location 1060.

FIG. 11 illustrates generally a block diagram 1100 of height upmixprocessing for a single channel input signal 1101. The input signal 1101can be divided into a horizontal-path signal and a height-path signal.In an example, the horizontal-path signal can be passed to a horizontalspeaker output 1102. The height-path signal can be received at a Delaymodule 1110. After a specified delay duration is applied to theheight-path signal, the delayed signal can be provided from the Delaymodule 1110 to a Decorrelation module 1120. The delay duration can beadjustable. Typical delay duration values can be in a range of about 5to 20 milliseconds to leverage the psycho-acoustic Haas Effect (a.k.a.“law of the first wave front”), such as to ensure that perceived soundsource localizations for transient input signals are maintained in thehorizontal speaker (see, e.g., FIG. 10). Other delay duration values cansimilarly be used.

For quasi-stationary signals having low auto-correlation, such asreverberation decay tails, an effect of the height upmix processingtechnique of FIG. 11 can be to expand the perceived sound localizationupward from the horizontal plane. In some examples, such as shown inFIG. 11, the Decorrelation module 1120 can apply a decorrelation filterto the height-path signal (and additionally or alternatively, to thehorizontal-path signal) to further reduce correlation between signals atthe height speaker output 1122 and at the horizontal speaker output1102. Such further decorrelation can enhance the perception or sensationof vertical extension.

FIG. 12 illustrates generally a block diagram of an example of theDecorrelation module 1120 from the example of FIG. 11. In this example,the decorrelation filter includes a Schroeder all-pass section 1200. Thefilter can have various adjustable parameters, including a delay oflength M, and a feedback gain g₁ having magnitude less than 1. In anexample, values for each of the magnitude of the feedback gain g₁ andfor the delay length can be about 0 to 10 milliseconds. Other values cansimilarly be used.

Some examples of systems that can perform virtual height upmixing areillustrated in FIGS. 13-16. In the examples, a horizontal channel inputsignal can be divided into multiple signal paths, including aheight-path signal and a horizontal-path signal, similarly to theexample of FIG. 11. The height-path signal can be forwarded to a virtualheight filter and then combined with an unprocessed, minimallyprocessed, or decorrelated version of the horizontal-path signal, suchas prior to optional horizontal-plane virtualization of the signal.

FIG. 13 illustrates generally a first height upmix processing example1300. The example 1300 includes a first input signal processing circuit1301 and an upmix processing circuit 1302. The first input signalprocessing circuit 1301 is configured to receive a horizontal channelinput signal and divide the signal to provide a height-path signal to anattenuation circuit (e.g., a parametric low-frequency shelvingattenuator circuit) and to provide a horizontal-path signal to a boostcircuit (e.g., a parametric low-frequency shelving boost circuit). In anexample, the attenuation and boost circuits can be quasi-complementarymeaning that an attenuation characteristic provided by the attenuatorcircuit can be opposed by a boost characteristic provided by the boostcircuit. In an example, the attenuation and boost characteristics canhave substantially equal but opposite values, however, unequal valuescan similarly be used. Outputs from the first signal processing circuit1301 can be provided to the upmix processing circuit 1302.

In the upmix processing circuit 1302, an attenuated signal from theattenuation circuit can be delayed using a delay circuit, and thenfurther processed using a Decorrelation module. In an example, theDecorrelation module decorrelates left and right channel signalcomponents, decorrelates height and horizontal channel signalcomponents, or decorrelates other signal components. Followingdecorrelation, the resulting decorrelated signals can be processed usinga virtual height filter and then mixed with the boosted horizontal-pathsignal from the boost circuit. The mixed signals can be optionallyprovided to a horizontal-plane virtualizer circuit for furtherprocessing, such as before being output to an amplifier, subsequentprocessor module, or loudspeaker.

In the example 1300 of FIG. 13, the Decorrelation module's left/rightand height/horizontal filter components can be combined into a singledecorrelation filter that can be realized, for example, using anall-pass filter, such as using the nested all-pass filter 800 from theexample of FIG. 8. In an example, the Decorrelation module can behelpful for mitigating timbre artifacts or sound coloration artifacts(sometimes referred to as “comb-filter” coloration) that can result fromdown-mixing a delayed height-path signal with an un-delayedhorizontal-path signal.

In an example, comb-filter coloration can be further mitigated byattenuating a height-path signal at lower frequencies, such as using ashelving equalization filter (e.g., using the attenuation circuit). Aboost shelving filter can be applied (e.g., using the boost circuit) tothe horizontal-path signal to help preserve an overall signal loudnesscharacteristic of the final combined output signal. Additionally, topreserve equal power across all signal frequencies, it can be helpfulfor the mix-down gain to be 0 dB, and for the attenuation and boost ofthe complementary shelving filters to be set to opposite-polarity values(e.g., +3 dB and −3 dB).

FIG. 14 illustrates generally a second height upmix processing example1400. The example 1400 includes a second input signal processing circuit1401 and the same upmix processing circuit 1302 from the example 1300 ofFIG. 13. In an example, one or more parameters of the upmix processingcircuit 1302 can be changed to accommodate signals from the second inputsignal processing circuit 1401. In the example 1400, thequasi-complementary attenuation and boost circuits from the first inputsignal processing circuit 1301 can be replaced with a single, all-passfilter and signal sum and difference operators. Sum and differencesignals can be obtained between the input signal and the output of afirst order or second order all-pass filter applied to the same inputsignal. To achieve attenuation and boost shelving effects, subsequentsums of the previous difference can be multiplied by attenuation andboost coefficients K_(A) and K_(B), respectively, and a previous sum canbe divided by a factor of two.

FIG. 15 illustrates generally a third height upmix processing example1500. The example 1500 includes a third input signal processing circuit1501 and the same upmix processing circuit 1302 from the example 1300 ofFIG. 13. In an example, one or more parameters of the upmix processingcircuit 1302 can be changed to accommodate signals from the third inputsignal processing circuit 1501. In the example 1500, thequasi-complementary attenuation and boost circuits from the first inputsignal processing circuit 1301 can be replaced with a single low-passfilter and sum and difference operators. In the example 1500, a sum anddifference can be obtained between the input signal and the output ofthe low-pass filter applied to the same input signal.

FIG. 16 illustrates generally a fourth height upmix processing example1600. The example 1600 includes a fourth input signal processing circuit1601 and the same upmix processing circuit 1302 from the example 1300 ofFIG. 13. In an example, one or more parameters of the upmix processingcircuit 1302 can be changed to accommodate signals from the fourth inputsignal processing circuit 1601. In the example 1600, thequasi-complementary attenuation and boost circuits from the first inputsignal processing circuit 1301 can be implemented using a parallelcombination of all-pass filters (“All-pass Filter 1” and “All-passFilter 2”) followed by sum and difference operators. Sum and differencesignals can be obtained between an output of All-pass Filter 1 and anoutput of All-pass Filter 2. To attain attenuation and boost shelvingeffects, subsequent sums of the previous difference multiplied byattenuation and boost coefficients K_(A) and K_(B), respectively, can beapplied, and a previous sum can be divided by a factor of two.

FIG. 17 illustrates generally first, second, and third examples 1701,1702, and 1703, of a virtual height upmix processor in a 5-channel inputsystem. The first example 1701 includes a signal flow diagram showing a5-channel input signal 1711 that includes signal components or channelsL, R, C, Ls, and Rs. Various hardware circuitry can be used to receivethe 5-channel input signal 1711, such as including discrete electricalor optical input paths to receive time-varying audio signal informationat an audio processor circuit.

In an example, one or more of the signal components or channels includesmetadata (e.g., analog or digital data encoded with audio signalinformation) with information about a localization for one or more ofthe same or other signal components or channels. In an example, thelocalization information can be provided via other means, such as usinga separate or dedicated hardware input to an audio processor circuit.The localization information can include an indication as to whichchannel(s) the localization information corresponds. In an example, thelocalization information includes azimuth and/or altitude information.The altitude information can include an indication of a localizationthat is above or below a reference plane.

In the first example 1701, the input signals are provided to an UpmixProcessor module 1712 that generates height signals Lh, Rh, Lsh, andRsh, such as based on information in the input signals. The UpmixProcessor module 1712 can include or use any of the systems shown in thefirst through fourth height upmix processing examples 1300, 1400, 1500,and 1600, from the examples of FIGS. 13, 14, 15, and 16 respectively.For example, the Upmix Processor module 1712 can be configured to spliteach input channel into a height-path signal to which a delay can beapplied, and a horizontal-path signal, such as with quasi-complementarylow-frequency attenuation and boost. In an example, the Upmix Processormodule 1712 can further be configured to pass the input signal 1711 (L,R, C, Ls, and Rs) to a first Mixer module 1715.

In the first example 1701, the four height signals generated by theUpmix Processor module 1712 can be provided to a Decorrelation module1713, and at least one or more of the four input signals can besubjected to a decorrelation filter. In an example, each of the fourinput signals can be subjected to a decorrelation filter that includesor uses a unique instance of a nested all-pass filter, such as thefilter 800 of FIG. 8. Other hardware filters or circuits can similarlybe used or applied to generate decorrelated signals, such as using aphase-shift or time-delay audio filter circuit. Following decorrelationprocessing by the Decorrelation module 1713, resulting decorrelatedsignals are provided to a Virtual Height Filter module 1714. In anexample, the Virtual Height Filter module 1714 includes or uses theHeight Virtualization module 375 from the example of FIG. 3 and appliessignal processing or filtering to the one or more decorrelated signals.

At the Virtual Height Filter module 1714, a front virtual height filtercan be applied to the height audio signal input pair (Lh, Rh), such asdescribed above in the discussion of FIG. 5, such as using an audioprocessor circuit. In an example, a rear virtual height filter can beapplied to the rear height input signal pair (Lsh, Rsh). In someexamples, the front and rear virtual height filters can be selectedbased on or using azimuth angle-specific HRTF data, such as can bemeasured relative to a direction of a C-channel (e.g., front centerchannel) speaker. In an example, the Virtual Height Filter module 1714and/or audio processor circuit generates a virtualized audio signal byfiltering the height audio signal input(s).

Following the Virtual Height Filter module 1714, filtered signals can beprovided to the Mixer module 1715, and the filtered height signals Lh,Rh, Lsh, and Rsh, can be down-mixed by the Mixer module 1715 into thecorresponding horizontal path signals (L, R, C, Ls and Rs) to produce a5-channel output signal 1719. The 5-channel output signal 1719 can beconfigured for use in audio reproduction using loudspeakers in a firstplane of a listener to produce audible information that is perceived bythe listener as including information outside of the first plane, forexample, above or below the first plane.

The second example 1702 illustrates a variation of the first example1701 that includes horizontal surround processing. The second example1702 can include a Horizontal Surround Processing module 1726 configuredto receive the 5-channel output signal from a Mixer module 1725, andprovide a down-mixed 2-channel output signal 1729 (e.g., a left andright stereo pair). The 2-channel output signal 1729 can be configuredfor use in audio reproduction using loudspeakers in a first plane of alistener to produce audible information that is perceived by thelistener as including information outside of the first plane, forexample, above or below the first plane.

In an example, the Horizontal Surround Processing module 1726 caninclude or use the Horizontal Plane Virtualization module 365 from theexample of FIG. 3 to virtualize or provide horizontally-located signalcomponents. In an example, the Horizontal Surround Processing module1726 includes a front-channel horizontal-plane virtualizer applied tothe left and right front signal pair (L, R), such as illustrated in theexample of FIG. 5, and a rear-channel horizontal-plane virtualizerapplied to the left and right side signal pair (Ls, Rs).

The third example 1703 illustrates a variation of the first example 1701that includes separately applied height surround processing andhorizontal surround processing. The third example 1703 can include aHorizontal Surround Processing module 1736 configured to receive the5-channel output signal from the Upmix Processor module 1712 and providea down-mixed 2-channel output signal (e.g., a left and right stereopair) to a Mixer module 1735. In an example, the Horizontal SurroundProcessing module 1736 can include or use the Horizontal PlaneVirtualization module 365 from the example of FIG. 3 to virtualize orprovide horizontally-located signal components. In an example, theHorizontal Surround Processing module 1736 includes a front-channelhorizontal-plane virtualizer applied to the left and right front signalpair (L, R), such as illustrated in the example of FIG. 5, and arear-channel horizontal-plane virtualizer applied to the left and rightside signal pair (Ls, Rs).

The third example 1703 can include a Height Surround Processing module1737 configured to receive output signals Lh, Rh, Lsh, and Rsh, from theVirtual Height Filter module 1714. The Height Surround Processing module1737 can further process and down-mix the four height signals from theVirtual Height Filter module 1714 to provide a down-mixed 2-channeloutput signal (e.g., a left and tight stereo pair). The respective2-channel output signals from the Horizontal Surround Processing module1736 and from the Height Surround Processing module 1737 can be combinedby a Mixer module 1735 to render a two-channel loudspeaker output signal1739. The 2-channel output signal 1739 can be configured for use inaudio reproduction using loudspeakers in a first plane of a listener toproduce audible information that is perceived by the listener asincluding information outside of the first plane, for example, above orbelow the first plane.

Various systems and machines can be configured to perform or carry outone or more of the signal processing tasks described herein. Forexample, any one or more of the Upmix modules, Decorrelation modules,Virtual Height Filter modules, Height Surround Processing modules,Horizontal Surround Processing modules, Mixer modules, or other modulesor processes, such as provided in the examples of FIGS. 9 and 17, can beimplemented using a general purpose or special, purpose-built machinethat performs the various processing tasks, such as using instructionsretrieved from a tangible, non-transitory, processor-readable medium.

FIG. 18 is a block diagram illustrating components of a machine 1800,according to some example embodiments, able to read instructions 1816from a machine-readable medium (e.g., a machine-readable storage medium)and perform any one or more of the methodologies discussed herein.Specifically, FIG. 18 shows a diagrammatic representation of the machine1800 in the example form of a computer system, within which theinstructions 1816 (e.g., software, a program, an application, an applet,an app, or other executable code) for causing the machine 1800 toperform any one or more of the methodologies discussed herein may beexecuted. For example, the instructions 1816 can implement modules orcircuits or components of FIGS. 5-7, and FIGS. 11-17, and so forth. Theinstructions 1816 can transform the general, non-programmed machine 1800into a particular machine programmed to carry out the described andillustrated functions in the manner described (e.g., as an audioprocessor circuit). In alternative embodiments, the machine 1800operates as a standalone device or can be coupled (e.g., networked) toother machines. In a networked deployment, the machine 1800 can operatein the capacity of a server machine or a client machine in aserver-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment.

The machine 1800 can comprise, but is not limited to, a server computer,a client computer, a personal computer (PC), a tablet computer, a laptopcomputer, a netbook, a set-top box (STB), a personal digital assistant(PDA), an entertainment media system or system component, a cellulartelephone, a smart phone, a mobile device, a wearable device (e.g., asmart watch), a smart home device (e.g., a smart appliance), other smartdevices, a web appliance, a network router, a network switch, a networkbridge, a headphone driver, or any machine capable of executing theinstructions 1816, sequentially or otherwise, that specify actions to betaken by the machine 1800. Further, while only a single machine 1800 isillustrated, the term “machine” shall also be taken to include acollection of machines 1800 that individually or jointly execute theinstructions 1816 to perform any one or more of the methodologiesdiscussed herein.

The machine 1800 can include or use processors 1810, such as includingan audio processor circuit, non-transitory memory/storage 1830, and I/Ocomponents 1850, which can be configured to communicate with each othersuch as via a bus 1802. In an example embodiment, the processors 1810(e.g., a central processing unit (CPU), a reduced instruction setcomputing (RISC) processor, a complex instruction set computing (CISC)processor, a graphics processing unit (GPU), a digital signal processor(DSP), an ASIC, a radio-frequency integrated circuit (RFIC), anotherprocessor, or any suitable combination thereof) can include, forexample, a circuit such as a processor 1812 and a processor 1814 thatmay execute the instructions 1816. The term “processor” is intended toinclude a multi-core processor 1812, 1814 that can comprise two or moreindependent processors 1812, 1814 (sometimes referred to as “cores”)that may execute the instructions 1816 contemporaneously. Although FIG.18 shows multiple processors 1810, the machine 1800 may include a singleprocessor 1812, 1814 with a single core, a single processor 1812, 1814with multiple cores (e.g., a multi-core processor 1812, 1814), multipleprocessors 1812, 1814 with a single core, multiple processors 1812, 1814with multiples cores, or any combination thereof, wherein any one ormore of the processors can include a circuit configured to apply aheight filter to an audio signal to render a processed or virtualizedaudio signal.

The memory/storage 1830 can include a memory 1832, such as a main memorycircuit, or other memory storage circuit, and a storage unit 1836, bothaccessible to the processors 1810 such as via the bus 1802. The storageunit 1836 and memory 1832 store the instructions 1816 embodying any oneor more of the methodologies or functions described herein. Theinstructions 1816 may also reside, completely or partially, within thememory 1832, within the storage unit 1836, within at least one of theprocessors 1810 (e.g., within the cache memory of processor 1812, 1814),or any suitable combination thereof, during execution thereof by themachine 1800. Accordingly, the memory 1832, the storage unit 1836, andthe memory of the processors 1810 are examples of machine-readablemedia.

As used herein, “machine-readable medium” means a device able to storethe instructions 1816 and data temporarily or permanently and mayinclude, but not be limited to, random-access memory (RAM), read-onlymemory (ROM), buffer memory, flash memory, optical media, magneticmedia, cache memory, other types of storage (e.g., erasable programmableread-only memory (EEPROM)), and/or any suitable combination thereof. Theterm “machine-readable medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,or associated caches and servers) able to store the instructions 1816.The term “machine-readable medium” shall also be taken to include anymedium, or combination of multiple media, that is capable of storinginstructions (e.g., instructions 1816) for execution by a machine (e.g.,machine 1800), such that the instructions 1816, when executed by one ormore processors of the machine 1800 (e.g., processors 1810), cause themachine 1800 to perform any one or more of the methodologies describedherein. Accordingly, a “machine-readable medium” refers to a singlestorage apparatus or device, as well as “cloud-based” storage systems orstorage networks that include multiple storage apparatus or devices. Theterm “machine-readable medium” excludes signals per se.

The I/O components 1850 may include a variety of components to receiveinput, provide output, produce output, transmit information, exchangeinformation, capture measurements, and so on. The specific I/Ocomponents 1850 that are included in a particular machine 1800 willdepend on the type of machine 1800. For example, portable machines suchas mobile phones will likely include a touch input device or other suchinput mechanisms, while a headless server machine will likely notinclude such a touch input device. It will be appreciated that the I/Ocomponents 1850 may include many other components that are not shown inFIG. 18. The I/O components 1850 are grouped by functionality merely forsimplifying the following discussion, and the grouping is in no waylimiting. In various example embodiments, the I/O components 1850 mayinclude output components 1852 and input components 1854. The outputcomponents 1852 can include visual components (e.g., a display such as aplasma display panel (PDP), a light emitting diode (LED) display, aliquid crystal display (LCD), a projector, or a cathode ray tube (CRT)),acoustic components (e.g., loudspeakers), haptic components (e.g., avibratory motor, resistance mechanisms), other signal generators, and soforth. The input components 1854 can include alphanumeric inputcomponents (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, or other pointinginstruments), tactile input components (e.g., a physical button, a touchscreen that provides location and/or force of touches or touch gestures,or other tactile input components), audio input components (e.g., amicrophone), and the like.

In further example embodiments, the I/O components 1850 can includebiometric components 1856, motion components 1858, environmentalcomponents 1860, or position components 1862, among a wide array ofother components. For example, the biometric components 1856 can includecomponents to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification,fingerprint identification, or electroencephalogram basedidentification), and the like, such as can influence a inclusion, use,or selection of a listener-specific or environment-specific impulseresponse or HRTF, for example. In an example, the biometric components1856 can include one or more sensors configured to sense or provideinformation about a detected location of the listener 110 in anenvironment. The motion components 1858 can include acceleration sensorcomponents (e.g., accelerometer), gravitation sensor components,rotation sensor components (e.g., gyroscope), and so forth, such as canbe used to track changes in the location of the listener 110. Theenvironmental components 1860 can include, for example, illuminationsensor components (e.g., photometer), temperature sensor components(e.g., one or more thermometers that detect ambient temperature),humidity sensor components, pressure sensor components (e.g.,barometer), acoustic sensor components (e.g., one or more microphonesthat detect reverberation decay times, such as for one or morefrequencies or frequency bands), proximity sensor or room volume sensingcomponents (e.g., infrared sensors that detect nearby objects), gassensors (e.g., gas detection sensors to detect concentrations ofhazardous gases for safety or to measure pollutants in the atmosphere),or other components that may provide indications, measurements, orsignals corresponding to a surrounding physical environment. Theposition components 1862 can include location sensor components (e.g., aGlobal Position System (GPS) receiver component), altitude sensorcomponents (e.g., altimeters or barometers that detect air pressure fromwhich altitude may be derived), orientation sensor components (e.g.,magnetometers), and the like.

Communication can be implemented using a wide variety of technologies.The I/O components 1850 can include communication components 1864operable to couple the machine 1800 to a network 1880 or devices 1870via a coupling 1882 and a coupling 1872 respectively. For example, thecommunication components 1864 can include a network interface componentor other suitable device to interface with the network 1880. In furtherexamples, the communication components 1864 can include wiredcommunication components, wireless communication components, cellularcommunication components, near field communication (NFC) components,Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components,and other communication components to provide communication via othermodalities. The devices 1870 can be another machine or any of a widevariety of peripheral devices (e.g., a peripheral device coupled via aUSB).

Moreover, the communication components 1864 can detect identifiers orinclude components operable to detect identifiers. For example, thecommunication components 1864 can include radio frequency identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as Universal Product Code (UPC) bar code,multi-dimensional bar codes such as Quick Response (QR) code, Azteccode, Data Matrix, Dataglyph, MaxiCode, PDF49, Ultra Code, UCC RSS-2Dbar code, and other optical codes), or acoustic detection components(e.g., microphones to identify tagged audio signals). In addition, avariety of information can be derived via the communication components1864, such as location via Internet Protocol (IP) geolocation, locationvia Wi-Fi® signal triangulation, location via detecting an NFC beaconsignal that may indicate a particular location, and so forth. Suchidentifiers can be used to determine information about one or more of areference or local impulse response, reference or local environmentcharacteristic, or a listener-specific characteristic.

In various example embodiments, one or more portions of the network 1880can be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the publicswitched telephone network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a network,another type of network, or a combination of two or more such networks.For example, the network 1880 or a portion of the network 1880 caninclude a wireless or cellular network and the coupling 1882 may be aCode Division Multiple Access (CDMA) connection, a Global System forMobile communications (GSM) connection, or another type of cellular orwireless coupling. In this example, the coupling 1882 can implement anyof a variety of types of data transfer technology, such as SingleCarrier Radio Transmission Technology (1xRTT), Evolution-Data Optimized(EVDO) technology, General Packet Radio Service (GPRS) technology,Enhanced Data rates for GSM Evolution (EDGE) technology, thirdGeneration Partnership Project (3GPP) including 3G, fourth generationwireless (4G) networks, Universal Mobile Telecommunications System(UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability forMicrowave Access (WiMAX), Long Term Evolution (LTE) standard, othersdefined by various standard-setting organizations, other long rangeprotocols, or other data transfer technology. In an example, such awireless communication protocol or network can be configured to transmitheadphone audio signals from a centralized processor or machine to aheadphone device in use by a listener.

The instructions 1816 can be transmitted or received over the network1880 using a transmission medium via a network interface device (e.g., anetwork interface component included in the communication components1864) and using any one of a number of well-known transfer protocols(e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions1816 can be transmitted or received using a transmission medium via thecoupling 1872 (e.g., a peer-to-peer coupling) to the devices 1870. Theterm “transmission medium” shall be taken to include any intangiblemedium that is capable of storing, encoding, or carrying theinstructions 1816 for execution by the machine 1800, and includesdigital or analog communications signals or other intangible media tofacilitate communication of such software.

Many variations of the concepts and examples discussed herein will beapparent to those skilled in the relevant arts. For example, dependingon the embodiment, certain acts, events, or functions of any of themethods, processes, or algorithms described herein can be performed in adifferent sequence, can be added, merged, or omitted (such that not alldescribed acts or events are necessary for the practice of the variousmethods, processes, or algorithms). Moreover, in some embodiments, actsor events can be performed concurrently, such as through multi-threadedprocessing, interrupt processing, or multiple processors or processorcores or on other parallel architectures, rather than sequentially. Inaddition, different tasks or processes can be performed by differentmachines and computing systems that can function together.

The various illustrative logical blocks, modules, methods, and algorithmprocesses and sequences described in connection with the embodimentsdisclosed herein can be implemented as electronic hardware, computersoftware, or combinations of both. To illustrate this interchangeabilityof hardware and software, various components, blocks, modules, andprocess actions are, in some instances, described generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. The described functionalitycan thus be implemented in varying ways for a particular application,but such implementation decisions should not be interpreted as causing adeparture from the scope of this document. Embodiments of the immersivespatial audio reproduction systems and methods and techniques describedherein are operational within numerous types of general purpose orspecial purpose computing system environments or configurations, such asdescribed above in the discussion of FIG. 18.

Various aspects of the invention can be used independently or together.For example, Aspect 1 can include or use subject matter (such as anapparatus, a system, a device, a method, a means for performing acts, ora device readable medium including instructions that, when performed bythe device, can cause the device to perform acts), such as can includeor use a method for providing virtualized audio information in athree-dimensional soundfield using loudspeakers arranged in a firstplane, wherein the virtualized audio information is perceived by alistener as including audible information in other than the first plane.In Aspect 1, the method can include receiving, using a first processorcircuit, at least one height audio signal, the at least one height audiosignal configured for use in audio reproduction using a loudspeaker thatis offset from the first plane, and receiving, using the first processorcircuit, localization information corresponding to the at least oneheight audio signal, the localization information including an azimuthparameter. Aspect 1 can further include selecting, using the firstprocessor circuit, a first virtual height filter using information aboutthe azimuth parameter, and generating a virtualized audio signal,including using the first processor circuit to apply the first virtualheight filter to the at least one height audio signal, wherein thevirtualized audio signal is configured for use in audio reproductionusing one or more loudspeakers in the first plane, and wherein when thevirtualized audio signal is reproduced using the one or moreloudspeakers it is perceived by a listener as including audibleinformation in other than the first plane. In an example, the firstplane of Aspect 1 corresponds to a horizontal plane of the one or moreloudspeakers used to reproduce the virtualized audio signal. In anexample, the first plane of Aspect 1 corresponds to a horizontal planeof the listener. In another example, horizontal planes of the listenerand the loudspeakers used to reproduce the virtualized audio signal arecoincident, and the first plane of Aspect 1 corresponds to thecoincident planes.

Aspect 2 can include or use, or can optionally be combined with thesubject matter of Aspect 1, to optionally include the generating thevirtualized audio signal includes generating the signal such that whenthe virtualized audio signal is reproduced using the one or moreloudspeakers, the virtualized audio signal is perceived by the listeneras including audible information that extends vertically upward ordownward from a horizontal plane of the loudspeakers to a second plane.

Aspect 3 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 or 2 to optionallyinclude the generating the virtualized audio signal includes generatingthe signal such that when the virtualized audio signal is reproducedusing the one or more loudspeakers, the virtualized audio signal isperceived by the listener as originating from an elevated or loweredsource relative to a horizontal plane of the loudspeakers.

Aspect 4 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 through 3 tooptionally include the generating the virtualized audio signal includesapplying horizontal-plane virtualization to the at least one heightaudio signal prior to applying the first virtual height filter.

Aspect 5 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 through 3 tooptionally include the generating the virtualized audio signal includesapplying horizontal-plane virtualization to the at least one heightaudio signal after applying the first virtual height filter.

Aspect 6 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 through 5 tooptionally include using an audio signal mixer circuit, combining thevirtualized audio signal with one or more other signals to beconcurrently reproduced using the one or more loudspeakers in the firstplane.

Aspect 7 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 through 6 tooptionally include the receiving the at least one height audio signalincludes receiving information about first and second height audiochannels intended for reproduction using different loudspeakers that areelevated relative to the first plane, wherein the first plane is ahorizontal plane of the listener, wherein the receiving the localizationinformation includes receiving respective azimuth parameters for thefirst and second height audio channels, wherein the selecting includesselecting different respective first and second virtual height filtersusing information about the respective azimuth parameters, and whereinthe generating includes using the first processor circuit to apply thefirst and second virtual height filters to the first and second heightaudio channels, respectively, to provide respective first and secondvirtualized audio signals, wherein when the first and second virtualizedaudio signals are reproduced using loudspeakers in the horizontal plane,the reproduced signals are perceived by the listener as includingaudible information in other than the horizontal plane.

Aspect 8 can include or use, or can optionally be combined with thesubject matter of Aspect 7, to optionally include the generatingincludes decorrelating the first and second height audio signals beforeapplying the first and second virtual height filters.

Aspect 9 can include or use, or can optionally be combined with thesubject matter of Aspect 7, to optionally include the respective azimuthparameters for the first and second height audio channels aresubstantially symmetrical azimuth angles, and wherein the selecteddifferent respective first and second virtual height filters include asum filter and a difference filter based on ipsilateral andcontralateral head-related transfer function data.

Aspect 10 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 through 9 tooptionally include the receiving the localization information furtherincludes receiving an altitude parameter, and wherein the selecting thefirst virtual height filter includes using information about the azimuthparameter and using information about the altitude parameter.

Aspect 11 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 through 10 tooptionally include the selecting the first virtual height filterincludes selecting a virtual height filter that is derived from ahead-related transfer function.

Aspect 12 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 1 through 11 tooptionally include the generating the virtualized audio signal furtherincludes using the first processor circuit to apply horizontal-planespatialization to the virtualized audio signal.

Aspect 13 can include or use, or can optionally be combined with thesubject matter of Aspect 12, to optionally include generatingspatially-enhanced audio signals for a horizontal plane, including usingthe first processor circuit to apply horizontal-plane spatialization toother audio signals intended for reproduction using loudspeakers in thehorizontal plane of the listener. Aspect 13 can further include mixingthe virtualized audio signal with the spatially-enhanced audio signalsto provide surround sound using the loudspeakers in the horizontal planeof the listener.

Aspect 14 can include, or can optionally be combined with the subjectmatter of one or any combination of Aspects 1 through 13 to include oruse, subject matter (such as an apparatus, a method, a means forperforming acts, or a machine readable medium including instructionsthat, when performed by the machine, that can cause the machine toperform acts), such as can include or use a system comprising means forreceiving a height audio information signal configured for use in audioreproduction using a loudspeaker that is outside of a first plane of alistener, means for receiving localization information corresponding tothe at least one height audio signal, the localization informationincluding an azimuth parameter, means for selecting a virtualized heightfilter using the azimuth parameter, and means for generating avirtualized height audio information signal using the selectedvirtualized height filter and the received height audio informationsignal, and for storing the virtualized height audio information signalon a non-transitory computer-readable medium, wherein the virtualizedheight audio information signal is configured for use in audioreproduction using a loudspeaker in the first plane of the listener.

Aspect 15 can include or use, or can optionally be combined with thesubject matter of Aspect 14 to optionally include the virtualized heightaudio information signal is configured for use in audio reproductionusing the loudspeaker in the first plane of the listener to provide anaudio image that extends vertically upward or downward from a horizontalplane of the loudspeaker used in the audio reproduction to a secondplane.

Aspect 16 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 14 or 15 tooptionally include the virtualized height audio information signal isconfigured for use in audio reproduction using the loudspeaker in thefirst plane of the listener to provide an audio image that originatesfrom a location that is offset vertically upward or downward from ahorizontal plane of the loudspeaker used in the audio reproduction.

Aspect 17 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 14 through 16 tooptionally include means for applying horizontal-plane virtualization tothe height audio information signal prior to generating the virtualizedheight audio information signal.

Aspect 18 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 14 through 17 tooptionally include means for combining the virtualized height audioinformation signal with one or more other signals to be concurrentlyreproduced using the loudspeaker in the first plane of the listener.

Aspect 19 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 14 through 18 tooptionally include means for decorrelating multiple channels of audioinformation in the height audio information signal to provide multipledecorrelated signals. In Aspect 19, the means for generating thevirtualized height audio information signal can include means forgenerating the virtualized height audio information signal using theselected virtualized height filter and at least one of the multipledecorrelated signals.

Aspect 20 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 14 through 19 tooptionally include the means for selecting the virtualized height filterusing the azimuth parameter includes means for selecting the virtualizedheight filter using an altitude parameter.

Aspect 21 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 14 through 20 tooptionally include means for generating the virtualized height filterusing information about a head-related transfer function.

Aspect 22 can include, or can optionally be combined with the subjectmatter of one or any combination of Aspects 1 through 21 to include oruse, subject matter (such as an apparatus, a method, a means forperforming acts, or a machine readable medium including instructionsthat, when performed by the machine, that can cause the machine toperform acts), such as can include or use an audio signal processingsystem configured to provide virtualized audio information in athree-dimensional soundfield using loudspeakers in a horizontal plane,wherein the virtualized audio information is perceived by a listener asincluding audible information in other than the horizontal plane. InAspect 22, the system includes an audio signal input configured toreceive at least one height audio signal, the at least one height audiosignal including audio signal information that is intended forreproduction using a loudspeaker that is elevated relative to a listener(e.g., relative to a horizontal plane associated with the listener), alocalization signal input configured to receive localization informationabout the at least one height audio signal, the localization informationincluding a first azimuth parameter, a memory circuit including one ormore virtual height filters, wherein each of the virtual height filtersis associated with one or more azimuth parameters, and an audio signalprocessor circuit configured to: retrieve a first virtual height filterfrom the memory circuit using the first azimuth parameter, and generatea virtualized audio signal by applying the first virtual height filterto the at least one height audio signal, wherein when the virtualizedaudio signal is reproduced using one or more loudspeakers in thehorizontal plane, the virtualized audio signal is perceived by thelistener as including audible information in other than the horizontalplane.

Aspect 23 can include or use, or can optionally be combined with thesubject matter of Aspect 22, to optionally include a decorrelationcircuit coupled to the audio signal input and configured to receive theat least one height audio signal, wherein the decorrelation circuit isconfigured to apply a decorrelation filter to one or more audio channelsincluded in the height audio signal.

Aspect 24 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 22 or 23 tooptionally include a horizontal-plane virtualization processor circuitconfigured to apply horizontal-plane virtualization to at least one ofthe height audio signal and the virtualized audio signal.

Aspect 25 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 22 through 24 tooptionally include a mixer circuit configured to combine the virtualizedaudio signal with one or more other signals to be concurrentlyreproduced using the same loudspeakers.

Aspect 26 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 22 through 25 tooptionally include the audio signal processor circuit includes ahead-related transfer function derivation circuit configured to derivethe first virtual height filter based on ipsilateral and contralateralhead-related transfer function information corresponding to thelistener.

Aspect 27 can include, or can optionally be combined with the subjectmatter of one or any combination of Aspects 1 through 16 to include oruse, subject matter (such as an apparatus, a method, a means forperforming acts, or a machine readable medium including instructionsthat, when performed by the machine, that can cause the machine toperform acts), such as can include or use a method for virtual heightprocessing of at least one height audio signal in a system with N audioinput channels, wherein the at least one height audio signal correspondsto one of the N audio input channels. In Aspect 27, the method caninclude selecting M channels for a down-mixed audio output from thesystem, wherein N and M are non-zero positive integers and wherein M isless than N, receiving, using an audio signal processor circuit,information about a virtual localization for the at least one heightaudio signal, the information about the virtual localization includingan azimuth parameter, and selecting, from a memory circuit, a virtualheight filter for use with the at least one height audio signal, theselecting based on the azimuth parameter. Aspect 27 can further includeproviding, using the audio signal processor circuit, a virtualized audiosignal using a virtualization processor circuit to process the at leastone height audio signal using the selected virtual height filter that isbased on the azimuth parameter, and mixing the virtualized audio signalwith other audio signal information from one or more of the selected Mchannels to provide an output signal.

Aspect 28 can include or use, or can optionally be combined with thesubject matter of Aspect 27 to optionally include deriving the virtualheight filter from a head-related transfer function corresponding to theazimuth parameter and/or an altitude parameter.

Aspect 29 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 27 or 28 tooptionally include deriving the virtual height filter using a ratio ofpower signals and based on the azimuth parameter.

Aspect 30 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 27 through 29 tooptionally include applying horizontal-plane spatialization to theoutput signal.

Aspect 31 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 27 through 30 tooptionally include the providing the virtualized audio signal includesapplying a decorrelation filter to at least one of multiple channels ofthe at least one height audio signal.

Aspect 32 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 27 through 31 tooptionally include wherein the at least one height audio signal includessignal information in each of two channels, wherein the receiving theinformation about the virtual localization includes receiving azimuthparameters respectively corresponding to the signal information in thetwo channels, wherein the azimuth parameters include substantiallysymmetrical virtual localization azimuth angles, and wherein theselecting the virtual height filter includes selecting a sum filter anda difference filter that are based on ipsilateral and contralateralhead-related transfer function data, respectively.

Aspect 33 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 27 through 32 tooptionally include the mixing includes mixing the signals to render atwo-channel headphone audio signal.

Aspect 34 can include, or can optionally be combined with the subjectmatter of one or any combination of Aspects 1 through 33 to include oruse, subject matter (such as an apparatus, a method, a means forperforming acts, or a machine readable medium including instructionsthat, when performed by the machine, that can cause the machine toperform acts), such as can include or use a method to vertically extendaudible artifact height in an audio signal that is reproduced usingloudspeakers provided substantially within a first plane. In Aspect 34,the method can include receiving, using a first processor circuit, afirst audio input signal, the audio input signal intended forreproduction using at least one of multiple loudspeakers provided in afirst plane of a listener, delaying the input audio signal and, usingthe first processor circuit, applying a virtual height filter to thefirst input audio signal to provide a virtualized height signal, andcombining, using the first processor circuit, the virtualized heightsignal and the audio input signal to provide a processed audio signal,wherein the processed audio signal is configured for reproduction usingone or more of the multiple loudspeakers provided in the first plane ofthe listener to provide an audible artifact that extends vertically fromthe first plane.

Aspect 35 can include or use, or can optionally be combined with thesubject matter of Aspect 34 to optionally include deriving the virtualheight filter from a head-related transfer function corresponding to anazimuth angle and an altitude angle associated with the verticallyextended audible artifact.

Aspect 36 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 34 or 35 tooptionally include the first audio input signal comprises information inat least two channels, and wherein the delaying applying the virtualheight filter to the first input audio signal further comprises applyinga decorrelation filter to at least one of the two channels prior to thecombining the virtualized height signal and the audio input signal toprovide the processed audio signal.

Aspect 37 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 34 through 36 tooptionally include applying a spectral correction filter to thevirtualized height signal to attenuate or amplify low frequencyinformation in the signal.

Aspect 38 can include, or can optionally be combined with the subjectmatter of one or any combination of Aspects 1 through 37 to include oruse, subject matter (such as an apparatus, a method, a means forperforming acts, or a machine readable medium including instructionsthat, when performed by the machine, that can cause the machine toperform acts), such as can include or use a method for virtualizationprocessing of an audio signal that includes two or more audioinformation channels. In Aspect 38, the method can include receiving,using a first processor circuit, an audio signal that includes multipleaudio information channels, applying, using the first processor circuit,a decorrelation filter to at least one of the multiple audio informationchannels to provide at least one filtered channel, and generating avirtualized audio signal, including using the first processor circuit toapply virtualization processing to the at least one filtered channel,the virtualization processing configured to adjust a listener-perceivedlocalization of audible information in the virtualized audio signal whenthe virtualized audio signal is provided to a listener usingloudspeakers or headphones.

Aspect 39 can include or use, or can optionally be combined with thesubject matter of Aspect 38 to optionally include the generating thevirtualized audio signal further comprises applying a virtual heightfilter to the at least one filtered channel, wherein the virtual heightfilter is derived from a head-related transfer function.

Aspect 40 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 38 or 39 tooptionally include the generating the virtualized audio signal furthercomprises applying a virtual height filter to the at least one filteredchannel, wherein the virtual height filter is derived from a power ratioof multiple head-related transfer functions.

Aspect 41 can include or use, or can optionally be combined with thesubject matter of Aspect 40 to optionally include deriving the virtualheight filter using magnitude information from first and secondhead-related transfer functions respectively associated with an audiosource that is offset from a listener in an azimuth direction and in anelevation direction.

Aspect 42 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 38 through 41 tooptionally include the applying the decorrelation filter includesapplying an all-pass filter to the at least one of the multiple audioinformation channels to provide the at least one filtered channel.

Aspect 43 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 38 through 42 tooptionally include the generating the virtualized audio signal includesapplying a head-related transfer function-based filter to adjust theperceived localization of an origin of audible information in thevirtualized audio signal when the virtualized audio signal is reproducedusing loudspeakers or headphones.

Aspect 44 can include, or can optionally be combined with the subjectmatter of one or any combination of Aspects 1 through 43 to include oruse, subject matter (such as an apparatus, a method, a means forperforming acts, or a machine readable medium including instructionsthat, when performed by the machine, that can cause the machine toperform acts), such as can include or use a system including means forreceiving an audio signal that includes multiple audio informationchannels, means for decorrelating the multiple audio informationchannels and providing at least one filtered channel, and means forgenerating a virtualized audio signal using the at least one filteredchannel, wherein the virtualized audio signal is configured for use inaudio reproduction using a loudspeaker in a first plane of a listener toproduce a listener-perceived localization of audible information outsideof the first plane.

Aspect 45 can include or use, or can optionally be combined with thesubject matter of Aspect 44 to optionally include the first plane is ahorizontal plane of the loudspeaker and the virtualized audio signal isconfigured for use in audio reproduction using the loudspeaker toproduce a listener-perceived localization of audible information thatextends above or below the horizontal plane.

Aspect 46 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 44 or 45 tooptionally include the first plane is a horizontal plane of theloudspeaker and the virtualized audio signal is configured for use inaudio reproduction using the loudspeaker to produce a listener-perceivedlocalization of audible information that originates above or below thehorizontal plane.

Aspect 47 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 44 through 46 tooptionally include the means for generating the virtualized audio signalincludes means for applying a head-related transfer function-basedvirtualization filter to the at least one filtered channel.

Aspect 48 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 44 through 47 tooptionally include means for applying horizontal-plane virtualization tothe filtered channel prior to generating the virtualized audio signal.

Aspect 49 can include or use, or can optionally be combined with thesubject matter of one or any combination of Aspects 44 through 48 tooptionally include means for combining the virtualized audio signal withone or more other signals to be concurrently reproduced using theloudspeaker in the first plane of the listener to producelistener-perceived localization of audible information inside the firstplane and outside the first plane.

Each of these non-limiting Aspects can stand on its own, or can becombined in various permutations or combinations with one or more of theother Aspects or examples provided herein.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated in this document, the terms “including” and “inwhich” are used as the plain-English equivalents of the respective terms“comprising” and “wherein.”

Conditional language used herein, such as, among others, “can,” “might,”“may,” “e.g.,” and the like, unless specifically stated otherwise, orotherwise understood within the context as used, is generally intendedto convey that certain embodiments include, while other embodiments donot include, certain features, elements and/or states. Thus, suchconditional language is not generally intended to imply that features,elements and/or states are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without author input or prompting, whether thesefeatures, elements and/or states are included or are to be performed inany particular embodiment.

While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it will beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As will berecognized, certain embodiments of the inventions described herein canbe embodied within a form that does not provide all of the features andbenefits set forth herein, as some features can be used or practicedseparately from others.

Moreover, although the subject matter has been described in languagespecific to structural features or methods or acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. A method for providing a three-dimensionalsoundfield using loudspeakers in a transverse plane, wherein thesoundfield includes audio information that is perceived by a listener asincluding information in an elevated plane, the method comprising:receiving a multiple-channel input including: at least two height audiosignals with information about one or more height audio sources, theheight audio signals configured for reproduction together to provide theone or more height audio sources using respective differently-positionedloudspeakers that are outside of the transverse plane, andtransverse-plane audio signals configured for reproduction usingrespective loudspeakers in the transverse plane; receiving respectiveazimuth parameters for each of the height audio signals; receivingrespective height parameters for each of the height audio signals;generating first virtualized audio signals by applying respectivedifferent inter-channel decorrelation processing filters and pairwisevirtualization processing to the at least two height audio signalsas-received, the pairwise virtualization processing using filters basedon the height parameters for the height audio signals, and thedecorrelation processing filters configured to enhance a localization,perceived by the listener, of one or more phantom sources in thesoundfield; and generating a multiple-channel output by combining thefirst virtualized audio signals with the transverse-plane audio signals,wherein combining the audio signals includes using the azimuthparameters for the height audio signals, and wherein generating themultiple-channel output includes generating second virtualized audiosignals by applying other virtualization processing to the firstvirtualized audio signals and the transverse-plane audio signalstogether, the other virtualization processing using horizontal planefilters based on the azimuth parameters for the height audio signals,and providing the multiple-channel output using the second virtualizedaudio signals, wherein the multiple-channel output comprises signalsconfigured for reproduction using respective different loudspeakers inthe transverse plane to provide the three-dimensional soundfield.
 2. Themethod of claim 1, wherein the one or more phantom sources comprise aportion of an audio image that originates from a location that is offsetvertically upward or downward from the transverse plane.
 3. The methodof claim 1, wherein generating the first virtualized audio signalsincludes using a head-related transfer function selected based on atleast one of the height parameters.
 4. The method of claim 1, whereingenerating the first virtualized audio signals includes applying therespective inter-channel decorrelation processing filters to the heightaudio signals as-received to provide decorrelated height signals andthen applying the pairwise virtualization processing to the decorrelatedheight signals.
 5. The method of claim 1, wherein generating the firstvirtualized audio signals includes applying the pairwise virtualizationprocessing to the height audio signals as-received to provideintermediate virtualized audio signals and then applying the respectiveinter-channel decorrelation processing filters to the intermediatevirtualized audio signals.
 6. The method of claim 1, wherein generatingthe multiple-channel output comprises mixing the first virtualized audiosignals and the transverse-plane audio signals together to provide theoutput.
 7. The method of claim 6, further comprising generating secondvirtualized audio signals by applying other virtualization processing tothe multiple-channel output, the other virtualization processing usinghorizontal plane filters based on the azimuth parameters for the heightaudio signals.
 8. The method of claim 1, wherein generating themultiple-channel output comprises: generating second virtualized audiosignals by applying other virtualization processing to the transverseplane audio signals; and combining the first virtualized audio signalswith the second virtualized audio signals using the azimuth parametersfor the height audio signals.
 9. The method of claim 1, wherein at leasttwo of the signals in the multiple-channel output comprise informationfrom the at least two height audio signals.
 10. An audio signalprocessing system configured to provide a three-dimensional soundfieldusing loudspeakers in a transverse plane, wherein the soundfieldincludes audio information that is perceived by a listener as includinginformation in an elevated plane, the system comprising: a first audiosignal input configured to receive at least two height audio signalsconfigured for reproduction together, to provide one or more heightaudio sources, using respective differently-positioned loudspeakers thatare elevated relative to the transverse plane; a second audio signalinput configured to receive transverse-plane audio signals configuredfor reproduction using respective loudspeakers in the transverse plane;a localization signal input configured to receive respective azimuth andheight parameters for each of the height audio signals; and an audiosignal processor circuit configured to: generate first virtualized audiosignals by applying respective different inter-channel decorrelationprocessing filters and pairwise virtualization processing to the heightaudio signals as-received, the pairwise virtualization processing usingfilters based on the height parameters for the height audio signals, andthe decorrelation processing filters configured to enhance alocalization, perceived by the listener, of one or more phantom sourcesin the soundfield; and generate a multiple-channel output by combiningthe first virtualized audio signals with the transverse-plane audiosignals, and applying other virtualization processing to the firstvirtualized audio signals and the transverse-plane audio signalstogether, the other virtualization processing using horizontal planefilters based on the azimuth parameters for the height audio signals,wherein combining the audio signals includes using the azimuthparameters for the height audio signals, wherein the multiple-channeloutput comprises signals configured for reproduction using respectivedifferent loudspeakers in the transverse plane to provide thethree-dimensional soundfield.
 11. The system of claim 10, wherein theaudio signal processor circuit comprises an audio signal mixer circuitconfigured to combine the first virtualized audio signals with thetransverse-plane audio signals to provide the multiple-channel output.12. The system of claim 10, wherein the audio signal processor circuitcomprises a traverse-plane surround sound processor configured toreceive the multiple-channel output and, in response, provide atwo-channel output representative of the three-dimensional soundfield.13. The system of claim 10, wherein the one or more phantom sourcescomprise a portion of the three-dimensional soundfield that is perceivedby the listener to originate from a location that is offset verticallyfrom the transverse plane.
 14. The system of claim 10, wherein the audiosignal processor circuit is configured to generate the first virtualizedaudio signals by applying the respective different inter-channeldecorrelation processing filters to the height audio signals as-receivedto provide decorrelated height signals and then applying the pairwisevirtualization processing to the decorrelated height signals.
 15. Thesystem of claim 10, wherein the audio signal processor circuit isconfigured to generate the first virtualized audio signals by applyingthe pairwise virtualization processing to the height audio signalsas-received to provide intermediate virtualized audio signals and thenapplying the respective different inter-channel decorrelation processingfilters to the intermediate virtualized audio signals.
 16. A system forproviding signals representative of a three-dimensional soundfield, thesignals configured for loudspeakers in a horizontal plane, wherein thesoundfield includes audio information that is perceived by a listener asincluding information outside of the horizontal plane, the systemcomprising: means for receiving a multiple-channel input including: atleast two height audio signals configured for reproduction together, toprovide one or more height audio sources, using respectivedifferently-positioned loudspeakers that are outside of the transverseplane, and transverse-plane audio signals configured for reproductionusing respective loudspeakers in the transverse plane; means forreceiving respective azimuth parameters for each of the height audiosignals; means for receiving respective height parameters for each ofthe height audio signals; means for generating first virtualized audiosignals by applying respective different inter-channel decorrelationprocessing filters and pairwise virtualization processing to the heightaudio signals, the pairwise virtualization processing using filtersbased on the height parameters for the height audio signals, and thedecorrelation processing filters configured to enhance a localization,perceived by the listener, of one or more phantom sources in thesoundfield; and means for generating a multiple-channel output bycombining the first virtualized audio signals with the transverse-planeaudio signals, and applying other virtualization processing to the firstvirtualized audio signals and the transverse-plane audio signalstogether, the other virtualization processing using horizontal planefilters based on the azimuth parameters for the height audio signals,wherein combining the audio signals includes using the azimuthparameters for the height audio signals, wherein the multiple-channeloutput comprises signals configured for reproduction using respectivedifferent loudspeakers in the transverse plane to provide thethree-dimensional soundfield.
 17. The system of claim 16, wherein themeans for generating the first virtualized audio signals includes: meansfor applying the inter-channel decorrelation processing filters to therespective height audio signals to provide decorrelated signals, andmeans for applying pairwise virtualization processing to thedecorrelated signals to provide the first virtualized audio signals. 18.The system of claim 16, wherein the means for generating the firstvirtualized audio signals includes: means for applying pairwisevirtualization processing to the height audio signals to provideintermediate virtualized audio signals, and means for applying theinter-channel decorrelation processing filters to the respectiveintermediate virtualized audio signals to provide the first virtualizedaudio signals.